Empathetic image selection

ABSTRACT

An approach is provided for selecting and displaying image(s). A user and user profile information corresponding to the user are identified. A sentiment of the user is identified by determining that user is in proximity to a digital picture frame in a room, receiving a measurement of ambient lighting of the room from a light sensor coupled to the digital picture frame, and determining an emotional state of the user based on the ambient lighting. Based on the user profile, an association is determined between the sentiment of the user and sentiment(s), which are determined to be conveyed by image(s). Based on the ambient lighting, the association between the sentiment of the user and the sentiment(s), and the image(s) conveying the sentiment(s), the image(s) are selected from multiple images. The selected image(s) are displayed on a display included in the digital picture frame.

This application is a continuation application claiming priority to Ser.No. 15/298,638 filed Oct. 20, 2016, now U.S. Pat. No. 9,817,625 issuedNov. 14, 2017.

BACKGROUND

The present invention relates to managing a presentation of images, andmore particularly to selecting and displaying images.

Photographs have the power to preserve a moment in time by capturinglight and storing the information it contains. The original process fortaking photographs was time and resource intensive and involved severalpieces of equipment to light the environment, capture the subjectmatter, and develop the image. As a result, relatively few people couldafford photographs for the first few decades following the invention.

As photographic technology progressed with the creation of film, flash,and instantaneous images, photography became more and more accessibleuntil nearly everyone could afford a personal camera that they could useto create their own pictures. The development process for these imageswas reduced to as little as an hour. The affordability, ease of use, andreduced development time led many people to create personal photo albumsto store their pictures. Creating these albums required time and effortin order to select the right grouping and ordering of picturesthroughout an album. These albums could then be easily accessed andshown to others, or simply reviewed individually in order to fondlyrecall people, places, and experiences in one's life.

The nature of photographic technology and the way in which we use it ona daily basis has been revolutionized in recent years. Digital phototechnology allows users to instantly create digital images that do notrequire development, which greatly reduces the cost of each image andallows users to afford many more pictures as compared to film. Due tothe widespread use and availability of the internet, users can nowupload, store, and access their photos from anywhere with internetconnectivity. Users no longer need bookcases full of albums that occupyvaluable space to store their images, and therefore the cost of storagehas also been greatly reduced. Finally, the mobile and social technologyrevolution has given people more incentive than ever to capture andshare images with others. All of these technological advances combinedhave encouraged people to upload about two billion new photos to theinternet each day, and this rate is increasing as more people gainaccess to mobile devices and internet connectivity.

SUMMARY

In one embodiment, the present invention provides a method of selectingand displaying images. The method includes a computer identifying a userand user profile information corresponding to the identified user. Themethod further includes the computer identifying a sentiment of theuser. The method further includes based on the user profile information,the computer determining an association between the identified sentimentof the user and one or more sentiments included in a plurality ofsentiments conveyed by a plurality of images. The method furtherincludes the computer determining that one or more images included inthe plurality of images convey the one or more sentiments. The methodfurther includes based on the association between the identifiedsentiment of the user and the one or more sentiments and the one or moreimages conveying the one or more sentiments, the computer selecting theone or more images from the plurality of images. The method furtherincludes the computer displaying the selected one or more images.

In another embodiment, the present invention provides a computer programproduct including a computer-readable storage device and acomputer-readable program code stored in the computer-readable storagedevice. The computer-readable program code includes instructions thatare executed by a central processing unit (CPU) of a computer system toimplement a method of selecting and displaying images. The methodincludes the computer system identifying a user and user profileinformation corresponding to the identified user. The method furtherincludes the computer system identifying a sentiment of the user. Themethod further includes based on the user profile information, thecomputer system determining an association between the identifiedsentiment of the user and one or more sentiments included in a pluralityof sentiments conveyed by a plurality of images. The method furtherincludes the computer system determining that one or more imagesincluded in the plurality of images convey the one or more sentiments.The method further includes based on the association between theidentified sentiment of the user and the one or more sentiments and theone or more images conveying the one or more sentiments, the computersystem selecting the one or more images from the plurality of images.The method further includes the computer system displaying the selectedone or more images.

In another embodiment, the present invention provides a computer systemincluding a central processing unit (CPU); a memory coupled to the CPU;and a computer-readable storage device coupled to the CPU. The storagedevice includes instructions that are executed by the CPU via the memoryto implement a method of selecting and displaying images. The methodincludes the computer system identifying a user and user profileinformation corresponding to the identified user. The method furtherincludes the computer system identifying a sentiment of the user. Themethod further includes based on the user profile information, thecomputer system determining an association between the identifiedsentiment of the user and one or more sentiments included in a pluralityof sentiments conveyed by a plurality of images. The method furtherincludes the computer system determining that one or more imagesincluded in the plurality of images convey the one or more sentiments.The method further includes based on the association between theidentified sentiment of the user and the one or more sentiments and theone or more images conveying the one or more sentiments, the computersystem selecting the one or more images from the plurality of images.The method further includes the computer system displaying the selectedone or more images.

Embodiments of the present invention automatically and quickly selectand display images in a user's photo collection, without requiring theuser to manually review the photo collection, identify the images theuser wants to view, and specify the order in which the images are to bedisplayed. The automated selection and display of images may be based onthe user's recent moods and/or recent experiences, without requiring theuser to manually select the images that correspond to an emotionalstate.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system for automatically selecting anddisplaying images, in accordance with embodiments of the presentinvention.

FIG. 2 is a flowchart of a process for automatically selecting anddisplaying images, where the process is implemented in the system ofFIG. 1, in accordance with embodiments of the present invention.

FIG. 3 is a flowchart of a process for creating user profile informationwhich is used in the process of FIG. 2, in accordance with embodimentsof the present invention.

FIGS. 4A-4C depict an example of automatically selecting and displayingimages, which utilizes the process of FIG. 2, in accordance withembodiments of the present invention.

FIG. 5 is a block diagram of a computer that is included in the systemof FIG. 1 and that implements the process of FIG. 2, in accordance withembodiments of the present invention.

DETAILED DESCRIPTION

Overview

Embodiments of the present invention automatically select and displayimages on a digital picture frame. In response to a user entering thevicinity of the digital picture frame, embodiments of the presentinvention automatically recognize the user, load the user's profile, andselect and display images on the digital picture frame, where theselection of the images is based on the user's profile, the user'scurrent sentiments, and the current environmental context relevant tothe user.

Embodiments of the present invention provide (1) automatic contenttagging based on the content of an image, (2) automatic sentimenttagging based on the sentiment typically associated with the content ofthe image, (3) automatic display of images associated with a specificuser in response to that user being within a predefined area inproximity to the digital picture frame, (4) incorporation of socialmedia data into the user's photo display preferences, (5) automaticupdating of user profile data based on social media content, (6)adaptive image selection based on the current mood of the user(s) in thepredefined area in proximity to the digital picture frame, and (7) mooddetermination of the environment of the digital picture frame based onaudio and video analysis.

Using known image management techniques presents a unique challengeregarding the management and timely display of photos. Currentlyavailable technology requires users to manually review their photocollections and identify which images the users want to view and inwhich order, in much the same way that people did with physical imagesthat were captured on film. Certain companies have begun to offermethods for automatically sorting images based on the image contentusing image recognition technology, which facilitates users' tasks ofsorting through their collections to find particular photos. If userswish to create slideshows of their images, however, the users mustmanually specify which images to display as well as the time of day, dayof the week, etc. on which to display them. Furthermore, users who wantto view different collections of images based on their mood and/orrecent experiences must manually select the images that correspond totheir emotional state and experiences. Since the amount of time andeffort required to perform the aforementioned manual review and manualselection of images is significant, many users fail to revisit thephotos they have taken, even if the users would enjoy viewing the photosagain.

System for Selecting and Displaying Images

FIG. 1 is a block diagram of a system 100 for automatically selectingand displaying images, in accordance with embodiments of the presentinvention. System 100 includes a computer 102 which executes asoftware-based empathetic image selection system 104 which analyzes andretrieves images based on preferences of a user and a sentiment of theuser, where the preferences are stored in a user profile. In oneembodiment, computer 102 is included in a digital picture frame (notshown) that displays selected images for viewing by one or more users.One or more users interact with empathetic image selection system 104via a user interface (UI) 105 (e.g., a user interface of a digitalpicture frame) to set preferences of the one or more users and manageuser profiles and images.

Empathetic image selection system 104 includes a remote useridentification and sentiment engine 106, a user and image managementengine 108, and a local user identification and sentiment engine 110.Remote user identification and sentiment engine 106 accesses imagesstored in an image repository 112. User and image management engine 108and local user identification and sentiment engine 110 access user datastored in a user data repository 114. Image repository 112 includesmetadata to determine the sentiment and tags of each image included inthe plurality of images stored in local storage and/or storage accessedvia the Internet or another computer network. User data repository 114stores user profile data, user preferences, and a last known sentimentof each user identified by empathetic image selection system 104.

Remote user identification and sentiment engine 106 is a software enginethat gathers sentiment and image data from remote sources (i.e., sourcesremote from computer 102, where the sources are in communication withcomputer 102 via a computer network (not shown)). In one embodiment, theremote sources include one or more social media services, one or morecalendar services or applications, one or more news and media services,and repositories of images (e.g., the user's storage locations providedby a network and image search engines).

Remote user identification and sentiment engine 106 includes a socialmedia classifier 116, a calendar classifier 118, a news & mediaclassifier 120, and image retrieval 122.

Social media classifier 116 is a software tool that gathers and analyzessocial media data that is authored by the user or that references theuser to identify a sentiment of the user.

Calendar classifier 118 is a software tool that gathers and analyzescalendar data managed by the user to identify the sentiment of the user.

News & media classifier 120 is a software tool that uses current eventdata accessed from online news and media services to determine anoverall sentiment at the current time.

Image retrieval 122 is a software tool that retrieves images fromexternal sources, including image search engines and the user's storagelocations provided by a network.

User and image management engine 108 is a software engine that managesuser profiles, analyzes image content and sentiment, and calculates thecurrent sentiment of the users within the vicinity of the frame. Userand image management engine 108 includes a profile manager 124, a user &image data comparator 126, a user empathy calculator 128, and an imagemetadata generator 130.

Profile manager 124 is a software tool that controls the preferences andother data of the user which is stored in user data repository 114. Auser can set the user's preferences controlled by profile manager 124,where the preferences include indications of images or categories ofimages the user likes, how often images are pulled from social media,etc.

User & image data comparator 126 is a software tool that determines thecurrent users identified by empathetic image selection system 104 andfinds commonalties among the user profiles of the current users.

User empathy calculator 128 is a software tool that uses local sensordata and the most recent remote data to determine the user's currentsentiment. User empathy calculator 128 calculates a final determinationof the sentiment of the user through an analysis of all the datacollected by empathetic image selection system 104, which selectsimage(s) for display, where the selected image(s) correspond to thesentiment of the user.

Image metadata generator 130 is a software tool that analyzes images todetermine their content and sentiment, which is stored in imagerepository 112 for future reference as image metadata.

Local user identification and sentiment engine 110 is a software enginethat identifies users within a proximity of a physical digital pictureframe which includes computer 102 and identifies the sentiments of theidentified users. Local user identification and sentiment engine 110updates the user data within the user data repository 114 and sends theresults to user and image management engine 108. Local useridentification and sentiment engine 110 includes a local videoclassifier 132, an audio classifier 134, a light feed classifier 136,and a weather classifier 138.

Local video classifier 132 is a software tool that identifies the userswho are in proximity to the digital picture frame that includes computer102 (e.g., identifies users who are in the room that includes thedigital picture frame) by using data from a video sensor (not shown)coupled to computer 102. Local video classifier 132 also identifies asentiment of the user by using data from the video sensor.

Audio classifier 134 is a software tool that identifies the users whoare in proximity to the digital picture frame (e.g., identifies theusers who are in the room in which the digital picture frame is located)through the use of data from an audio sensor (not shown) coupled tocomputer 102. Audio classifier 134 also identifies a sentiment of theuser by using the data from the audio sensor.

Light feed classifier 136 is a software tool that uses data from a lightsensor (not shown) coupled to computer 102 to measure the lighting inthe room or other environment that includes computer 102. Light feedclassifier 136 uses the measurement of the lighting to determine themood at the current time. For example, a measurement indicating asignificant amount of bright light in a room may indicate a lighter,happier mood associated with the room, whereas another measurementindicating dim light in the room may indicate a more somber mood.

Weather classifier 138 is a software tool that ingests data from aweather sensor (not shown) coupled to computer 102 to determine the moodat the current time. For example, if weather classifier 138 receivesdata from the weather sensor that indicates that it is raining outside,then the mood may be determined to be somber compared to a happier mooddetermined in response to weather classifier 138 receiving dataindicating that is sunny outside.

Remote user identification and sentiment engine 106 gathers sentimentand image data from the remote sources that provide social media data140, calendar data 142, news and media data 144, and other data 146.Social media data 140 includes data from social media services,including messages and other content posted by the user on a socialnetwork website. Calendar data 142 includes data from calendar servicesand/or applications, where the data includes information about andschedules of events of interest to the user. News and media data 144includes data from news and media services. Other data 146 includes datathat is not included in social media data 140, calendar data 142, andnews and media data 144. For example, other data 146 may include imagesfrom an image search service.

Local user identification and sentiment engine 110 receives data fromone or more sensors (not shown) coupled to computer 102 (e.g., sensorsmounted on a digital picture frame that includes computer 102), wherethe data includes a combination of local video sensor data 148, localaudio sensor data 150, local light sensor data 152, and local weathersensor data 154.

Local video sensor data 148 is provided by camera(s) (not shown) andother video identification software and/or hardware (not shown) whichgathers optical (i.e., visual) data about users in proximity to computer102 or the digital picture frame (not shown) that includes computer 102(e.g., optical data about users in the room that includes the digitalpicture frame). Local video classifier 132 uses local video sensor data148 to identify a user who is in proximity to computer 102 or thedigital picture frame that includes computer 102 and/or determine asentiment of the user.

Local audio sensor data 150 is provided by microphone(s) (not shown) andother audio identification software and/or hardware (not shown), whichgathers audio data about the users in proximity to computer 102 or adigital picture frame that includes computer 102 (e.g., audio data aboutusers in the room that includes the digital picture frame). Audioclassifier 134 uses local audio sensor data 150 to identify a user whois in proximity to computer 102 or the digital picture frame thatincludes computer 102 and/or determine a sentiment of the user.

Local light sensor data 152 is provided by an ambient light sensor (notshown) coupled to computer 102. Light feed classifier 136 uses locallight sensor data 152 to determine the mood or atmosphere of thelocation (e.g., room) which includes computer 102 or the digital pictureframe that includes computer 102.

In one embodiment, local weather sensor data 154 is pulled from one ormore weather reporting services by a weather sensor (not shown) coupledto computer 102. Weather classifier 138 uses local weather sensor data154 to determine the current weather, which is subsequently used todetermine a mood or atmosphere of the location (e.g., room) whichincludes computer 102 or the digital picture frame that includescomputer 102.

The functionality of the components shown in FIG. 1 is described in moredetail in the discussion of FIG. 2, FIG. 3, and FIG. 5 presented below.

Process for Selecting and Displaying Images

FIG. 2 is a flowchart of a process for automatically selecting anddisplaying images, where the process is implemented in the system ofFIG. 1, in accordance with embodiments of the present invention. Theprocess of FIG. 2 begins at step 200. In step 202, empathetic imageselection system 104 (see FIG. 1) identifies a user in a proximity to(i.e., within a predefined distance from) computer 102 or a digitalpicture frame that includes computer 102 (see FIG. 1). Step 202 alsoincludes empathetic image selection system 104 (see FIG. 1) identifyinguser profile information corresponding to the identified user. In oneembodiment, local user identification and sentiment engine 110 (seeFIG. 1) executing local video classifier 132 (see FIG. 1) and audioclassifier 134 (see FIG. 1) performs step 202 to analyze local videosensor data 148 (see FIG. 1) and local audio sensor data 150 (see FIG.1), respectively, to determine whether there is a match betweencharacteristics of video and audio data describing a user andcharacteristics that had been stored in user data repository 114 (seeFIG. 1) prior to the process of FIG. 2 (e.g., by step 308 in FIG. 3, asdiscussed below).

In step 204, empathetic image selection system 104 (see FIG. 1)identifies a sentiment of the user. The sentiment is a current emotionalstate of the user, another current cognitive state of the user, or aninterest of the user. In one embodiment, empathetic image selectionsystem 104 (see FIG. 1) performs step 204 utilizing user empathycalculator 128 (see FIG. 1), which uses an analysis of remote datasources performed by remote user identification and sentiment engine 106(see FIG. 1) and an analysis of local data sources performed by localuser identification and sentiment engine 110 (see FIG. 1).

The remote data sources include social media data 140 (see FIG. 1),calendar data 142 (see FIG. 1), news and media data 144 (see FIG. 1),and other data 146 (see FIG. 1) (e.g., a set of images resulting fromsearches the user performed using an image search engine).

The local data sources include local video sensor data 148 (see FIG. 1),local audio sensor data 150 (see FIG. 1), local light sensor data 152(see FIG. 1), and local weather sensor data 154 (see FIG. 1).

In step 206, based on the user profile information identified in step202, empathetic image selection system 104 (see FIG. 1) determines anassociation between (i) the sentiment of the user identified in step 204and (ii) one or more sentiments included in a plurality of sentimentsconveyed by a plurality of images. In one embodiment, determining theassociation includes empathetic image selection system 104 (see FIG. 1)retrieving associations from a table or other data structure thatindicate that a user has indicated a preference to view images conveyinga particular sentiment in response to the user having an emotional statethat describes the same sentiment or a different sentiment. Theretrieved association may also indicate a user's preference to viewimages that include a particular content in response to the user havinga particular sentiment. As one example, the association determined instep 206 indicates that the user prefers to view images conveying thesentiment of happiness in response to the user having the emotionalstate of feeling happy or sad. As another example, the associationdetermined in step 206 indicates that the user prefers to view imagesthat include favorite people of the user in response to the user havingthe emotional state of feeling happy, sad, or mad. In one embodiment,user empathy calculator 128 (see FIG. 1) can perform step 206.

In step 208, empathetic image selection system 104 (see FIG. 1)determines that one or more images included the plurality of imagesconvey the one or more sentiments which are associated with thesentiment identified in step 204 according to the association determinedin step 206. In one embodiment, user empathy calculator 128 (see FIG. 1)can perform step 208.

In step 210, based on (i) the association determined in step 206 and(ii) the one or more images determined in step 208, empathetic imageselection system 104 (see FIG. 1) selects the one or more images fromthe plurality of images. In one embodiment, user empathy calculator 128(see FIG. 1) can perform step 210.

In step 212, empathetic image selection system 104 (see FIG. 1) displaysthe one or more images selected in step 210 for viewing by the user.After step 212, the process of FIG. 2 ends at step 214.

In an alternate embodiment, step 204 identifies a mood of an environmentin proximity to the digital picture frame that includes computer 102(see FIG. 1), step 206 determines an association between the identifiedmood of the environment, and step 210 selects one or more imagesconveying one or more sentiments that correspond to the mood of theenvironment based on the association determined in step 206.

In another alternate embodiment, local user identification and sentimentengine 110 (see FIG. 1) determines that the user and one or moreadditional users are in proximity to computer 102 (see FIG. 1) or adigital picture frame that includes computer 102 (see FIG. 1). Inresponse to the user and the one or more additional users being inproximity to computer 102 (see FIG. 1) or the digital picture frame thatincludes computer 102 (see FIG. 1), user empathy calculator 128 (seeFIG. 1) identifies the sentiment of the user in step 204 and identifiesone or more sentiments (i.e., one or more emotional or other cognitivestates) of the one or more additional users, respectively. Based on anapplication of steps 206 and 208 to the aforementioned user and the oneor more additional users, in step 210, empathetic image selection system104 (see FIG. 1) selects image(s) that convey a sentiment thatcorresponds to the sentiments of the majority of the users who weredetermined to be in proximity to computer 102 (see FIG. 1) or thedigital picture frame.

Creating User Profile Information

FIG. 3 is a flowchart of a process for creating user profile informationwhich is used in the process of FIG. 2, in accordance with embodimentsof the present invention. The process of FIG. 3 precedes the process ofFIG. 2 and starts at step 300. In step 302, a digital picture frame(also known as “the frame”) that includes computer 102 (see FIG. 1)powers on for the first time and displays instructions for connecting tothe frame and for starting an initialization process.

In step 304, empathetic image selection system 104 (see FIG. 1) beingexecuted by the digital picture frame receives input from a user viauser interface 105 (see FIG. 1) to connect to the frame and continue theinitialization process.

In step 306, empathetic image selection system 104 (see FIG. 1) beingexecuted by the digital picture frame receives an administrator profilecreated by the user and properties of the frame which are set by theuser via profile manager 124 (see FIG. 1).

In step 308, empathetic image selection system 104 (see FIG. 1) beingexecuted by the digital picture frame receives a user profile created bythe user via user interface 105 (see FIG. 1). Step 308 also includesempathetic image selection system 104 (see FIG. 1) registering voicerecognition and facial recognition of the user and storing the voice andfacial recognition data in user data repository 114. In one embodiment,profile manager 124 (see FIG. 1) can perform step 308.

In step 310, video and audio recording hardware features included in thedigital picture frame take several images of the user and make audiorecordings of the user's speech for a subsequent matching of thecharacteristics of the images and audio recordings to local video sensordata 148 (see FIG. 1) and local audio sensor data 150 (see FIG. 1),where the matching is a basis for identifying a user who is in proximityto the digital picture frame. Also in step 310, empathetic imageselection system 104 (see FIG. 1) stores the characteristics of theimages of the user and the characteristics of the audio recordings ofthe user's speech in user data repository 114 (see FIG. 1). In oneembodiment, profile manager 124 (see FIG. 1) performs the storing of thecharacteristics in step 310.

In step 312, empathetic image selection system 104 (see FIG. 1) beingexecuted by the digital picture frame receives login information of oneor more social media accounts of the user and stores the logininformation in user data repository 114 (see FIG. 1). In one embodiment,profile manager 124 (see FIG. 1) performs step 312.

In step 314, empathetic image selection system 104 (see FIG. 1) beingexecuted by the digital picture frame receives user-specified localand/or online data repositories and subsequently, the digital pictureframe, as instructed by empathetic image selection system 104 (see FIG.1), pulls images from the user-specified data repositories and storesthe pulled images in image repository 112 (see FIG. 1). In oneembodiment, image retrieval 122 (see FIG. 1) performs step 314.

In step 316, empathetic image selection system 104 (see FIG. 1) storesuser preferences in user data repository 114 (see FIG. 1). The userpreferences include (1) the content of images that the user prefers toview in response to the user having a specified sentiment (i.e., aspecified emotional or other cognitive state), (2) the sentimentconveyed by images that the user prefers to detect in response to theuser having a specified sentiment, (3) how often the digital pictureframe is permitted by the user to pull images from social media in step314, etc. In one embodiment, profile manager 124 (see FIG. 1) performsstep 316.

In step 318, empathetic image selection system 104 (see FIG. 1) analyzesthe images pulled in step 314 and tags the pulled images with metadatathat specifies the content of the images and the respective sentimentsconveyed by the images. Subsequent to step 318, empathetic imageselection system 104 (see FIG. 1) stores the metadata tags in imagerepository 112 (see FIG. 1). In one embodiment, image metadata generator130 (see FIG. 1) performs step 318. After step 318, the process of FIG.3 ends at step 320.

Example

FIGS. 4A-4C depict a six-step example of automatically selecting anddisplaying images, which utilizes the process of FIG. 2, in accordancewith embodiments of the present invention. The steps shown in FIGS.4A-4C are performed subsequent to the process of FIG. 3. In step (1) inFIG. 4A, a digital picture frame 400 identifies a user 402 by using avideo camera 404 and an audio recorder 406. The digital picture frame400 includes computer 102 (see FIG. 1). Video camera 404 detects theface of user 402 and records an image of the face of user 402, where theimage of the face is part of local video sensor data 148 (see FIG. 1).Local video classifier 132 (see FIG. 1) analyzes the characteristics ofthe image of the face of user 402. Audio recorder 406 detects andrecords the speech of user 402, where the recorded speech is part oflocal audio sensor data 150 (see FIG. 1). Audio classifier 134 (seeFIG. 1) analyzes the characteristics of the recorded speech of user 402.Local user identification and sentiment engine 110 (see FIG. 1) uses theaforementioned analyses by local video classifier 132 (see FIG. 1) andaudio classifier 134 (see FIG. 1) to match the characteristics of theimage of the face and the recorded speech to characteristics of a userpreviously stored in user data repository 114 (see FIG. 1). By matchingthe characteristics, local user identification and sentiment engine 110(see FIG. 1) identifies the user. Using the identification of the user,profile manager 124 (see FIG. 1) identifies user profile information 408stored in user data repository 114 (see FIG. 1), where the user profileinformation 408 corresponds to the identified user. The identified userprofile information 408 includes user preferences. Step (1) in FIG. 4Ais included in step 202 in FIG. 2.

In step (2) in FIG. 4A, video camera 404 gathers visual data about thefacial expression on the face of user 402. Local video classifier 132(see FIG. 1) analyzes the gathered visual data and determines that thefacial expression of user 402 indicates happiness. Social mediaclassifier 116 (see FIG. 1) gathers and analyzes social media data thatis authored by the user, including a social media post 410 whose contentindicates that the user is happy. User empathy calculator 128 (seeFIG. 1) uses the aforementioned analyses by local video classifier 132(see FIG. 1) and social media classifier 116 (see FIG. 1) to make afinal determination that the current sentiment of the user is anemotional state of happiness. Step (2) in FIG. 4A is included in step204 in FIG. 2.

In step (3) in FIG. 4B, user empathy calculator 128 (see FIG. 1) indigital picture frame 400 retrieves information from a mood preferencestable 412 included in user profile information 408. The informationretrieved from mood preferences table 412 indicates an associationbetween the sentiment of the user and sentiments conveyed by imagesstored in image repository 112 (see FIG. 1). Mood preferences table 412had been previously input by user 402 via user interface 105 (seeFIG. 1) and had been stored by profile manager 124 (see FIG. 1) in step316 (see FIG. 3). Mood preferences table 412 indicates the user'spreference for viewing happy images in response to the user feelinghappy or sad and for viewing images that include at least one of theuser's favorite people in response to the user feeling happy, said, ormad. Step (3) in FIG. 4B is included in step 206 in FIG. 2.

In step (4) in FIG. 4C, based on the determination in step (2) (see FIG.4A) that the user is happy and based on the information in moodpreferences table 412 (see FIG. 4B) retrieved in step (3) (see FIG. 4B)that indicates that the user who is feeling happy prefers to see happyimages and images that include the user's favorite people, user empathycalculator 128 (see FIG. 1) searches a plurality of images 414 stored inimage repository 112 (see FIG. 1) and determines that images 416 conveya feeling of happiness and/or include at least one of the user'sfavorite people. Images 416 includes an image 416-1 which conveyshappiness, an image 416-2 which includes one of the favorite people ofuser 402, and an image 416-3 which conveys happiness and which includestwo of the favorite people of user 402. Step (4) in FIG. 4C is includedin step 208 in FIG. 2.

In step (5) in FIG. 4C, user empathy calculator 128 (see FIG. 1) selectsimages 416-1, 416-2, and 416-3 based on the mood preferences table 412(see FIG. 4B) that associates the user's feeling of happiness withimages that convey happiness and images that include at least one of theuser's favorite people and based on image 416-1 conveying happiness,image 416-2 including at least one of the user's favorite people, andimage 416-3 conveying happiness and including at least one of the user'sfavorite people. Step (5) in FIG. 4C is included in step 210 in FIG. 2.

In step (6) in FIG. 4C, digital picture frame 400 displays images 416-1,416-2, and 416-3 so that user 402 views the displayed images. Step (6)in FIG. 4C is included in step 212 in FIG. 2.

Computer System

FIG. 5 is a block diagram of a computer that is included in the systemof FIG. 1 and that implements the process of FIG. 2, in accordance withembodiments of the present invention. Computer 102 is a computer systemthat generally includes a central processing unit (CPU) 502, a memory504, an input/output (I/O) interface 506, and a bus 508. Computer 102 iscoupled to I/O devices 510 and a computer data storage unit 512. CPU 502performs computation and control functions of computer 102, includingexecuting instructions included in program code 514 for empathetic imageselection system 104 (see FIG. 1) to perform a method of automaticallyselecting and displaying one or more images, where the instructions areexecuted by CPU 502 via memory 504. CPU 502 may include a singleprocessing unit, or be distributed across one or more processing unitsin one or more locations (e.g., on a client and server).

Memory 504 includes a known computer readable storage medium, which isdescribed below. In one embodiment, cache memory elements of memory 504provide temporary storage of at least some program code (e.g., programcode 514) in order to reduce the number of times code must be retrievedfrom bulk storage while instructions of the program code are executed.Moreover, similar to CPU 502, memory 504 may reside at a single physicallocation, including one or more types of data storage, or be distributedacross a plurality of physical systems in various forms. Further, memory504 can include data distributed across, for example, a local areanetwork (LAN) or a wide area network (WAN).

I/O interface 506 includes any system for exchanging information to orfrom an external source. I/O devices 510 include any known type ofexternal device, including a display device, keyboard, etc. Bus 508provides a communication link between each of the components in computer102, and may include any type of transmission link, includingelectrical, optical, wireless, etc.

I/O interface 506 also allows computer 102 to store information (e.g.,data or program instructions such as program code 514) on and retrievethe information from computer data storage unit 512 or another computerdata storage unit (not shown). Computer data storage unit 512 includes aknown computer-readable storage medium, which is described below. In oneembodiment, computer data storage unit 512 is a non-volatile datastorage device, such as a magnetic disk drive (i.e., hard disk drive) oran optical disc drive (e.g., a CD-ROM drive which receives a CD-ROMdisk).

Memory 504 and/or storage unit 512 may store computer program code 514that includes instructions that are executed by CPU 502 via memory 504to automatically select and display one or more images. Although FIG. 5depicts memory 504 as including program code 514, the present inventioncontemplates embodiments in which memory 504 does not include all ofcode 514 simultaneously, but instead at one time includes only a portionof code 514.

Further, memory 504 may include an operating system (not shown) and mayinclude other systems not shown in FIG. 5.

Storage unit 512 and/or one or more other computer data storage units(not shown) that are coupled to computer 102 may store image repository112 (see FIG. 1), user data repository 114 (see FIG. 1), social mediadata 140 (see FIG. 1), calendar data 142 (see FIG. 1), news and mediadata 144 (see FIG. 1), other data 146 (see FIG. 1), local video sensordata 148 (see FIG. 1), local audio sensor data 150 (see FIG. 1), locallight sensor data 152 (see FIG. 1), and local weather sensor data 154(see FIG. 1).

As will be appreciated by one skilled in the art, in a first embodiment,the present invention may be a method; in a second embodiment, thepresent invention may be a system; and in a third embodiment, thepresent invention may be a computer program product.

Any of the components of an embodiment of the present invention can bedeployed, managed, serviced, etc. by a service provider that offers todeploy or integrate computing infrastructure with respect toautomatically selecting and displaying one or more images. Thus, anembodiment of the present invention discloses a process for supportingcomputer infrastructure, where the process includes providing at leastone support service for at least one of integrating, hosting,maintaining and deploying computer-readable code (e.g., program code514) in a computer system (e.g., computer 102) including one or moreprocessors (e.g., CPU 502), wherein the processor(s) carry outinstructions contained in the code causing the computer system toautomatically select and display one or more images. Another embodimentdiscloses a process for supporting computer infrastructure, where theprocess includes integrating computer-readable program code into acomputer system including a processor. The step of integrating includesstoring the program code in a computer-readable storage device of thecomputer system through use of the processor. The program code, uponbeing executed by the processor, implements a method of automaticallyselecting and displaying one or more images.

While it is understood that program code 514 for automatically selectingand displaying one or more images may be deployed by manually loadingdirectly in client, server and proxy computers (not shown) via loading acomputer-readable storage medium (e.g., computer data storage unit 512),program code 514 may also be automatically or semi-automaticallydeployed into computer 102 by sending program code 514 to a centralserver or a group of central servers. Program code 514 is thendownloaded into client computers (e.g., computer 102) that will executeprogram code 514. Alternatively, program code 514 is sent directly tothe client computer via e-mail. Program code 514 is then either detachedto a directory on the client computer or loaded into a directory on theclient computer by a button on the e-mail that executes a program thatdetaches program code 514 into a directory. Another alternative is tosend program code 514 directly to a directory on the client computerhard drive. In a case in which there are proxy servers, the processselects the proxy server code, determines on which computers to placethe proxy servers' code, transmits the proxy server code, and theninstalls the proxy server code on the proxy computer. Program code 514is transmitted to the proxy server and then it is stored on the proxyserver.

Another embodiment of the invention provides a method that performs theprocess steps on a subscription, advertising and/or fee basis. That is,a service provider, such as a Solution Integrator, can offer to create,maintain, support, etc. a process of automatically selecting anddisplaying one or more images. In this case, the service provider cancreate, maintain, support, etc. a computer infrastructure that performsthe process steps for one or more customers. In return, the serviceprovider can receive payment from the customer(s) under a subscriptionand/or fee agreement, and/or the service provider can receive paymentfrom the sale of advertising content to one or more third parties.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) (memory 504 and computer data storageunit 512) having computer readable program instructions 514 thereon forcausing a processor (e.g., CPU 502) to carry out aspects of the presentinvention.

The computer readable storage medium can be a tangible device that canretain and store instructions (e.g., program code 514) for use by aninstruction execution device (e.g., computer 102). The computer readablestorage medium may be, for example, but is not limited to, an electronicstorage device, a magnetic storage device, an optical storage device, anelectromagnetic storage device, a semiconductor storage device, or anysuitable combination of the foregoing. A non-exhaustive list of morespecific examples of the computer readable storage medium includes thefollowing: a portable computer diskette, a hard disk, a random accessmemory (RAM), a read-only memory (ROM), an erasable programmableread-only memory (EPROM or Flash memory), a static random access memory(SRAM), a portable compact disc read-only memory (CD-ROM), a digitalversatile disk (DVD), a memory stick, a floppy disk, a mechanicallyencoded device such as punch-cards or raised structures in a groovehaving instructions recorded thereon, and any suitable combination ofthe foregoing. A computer readable storage medium, as used herein, isnot to be construed as being transitory signals per se, such as radiowaves or other freely propagating electromagnetic waves, electromagneticwaves propagating through a waveguide or other transmission media (e.g.,light pulses passing through a fiber-optic cable), or electrical signalstransmitted through a wire.

Computer readable program instructions (e.g., program code 514)described herein can be downloaded to respective computing/processingdevices (e.g., computer 102) from a computer readable storage medium orto an external computer or external storage device (e.g., computer datastorage unit 512) via a network (not shown), for example, the Internet,a local area network, a wide area network and/or a wireless network. Thenetwork may comprise copper transmission cables, optical transmissionfibers, wireless transmission, routers, firewalls, switches, gatewaycomputers and/or edge servers. A network adapter card (not shown) ornetwork interface (not shown) in each computing/processing devicereceives computer readable program instructions from the network andforwards the computer readable program instructions for storage in acomputer readable storage medium within the respectivecomputing/processing device.

Computer readable program instructions (e.g., program code 514) forcarrying out operations of the present invention may be assemblerinstructions, instruction-set-architecture (ISA) instructions, machineinstructions, machine dependent instructions, microcode, firmwareinstructions, state-setting data, or either source code or object codewritten in any combination of one or more programming languages,including an object oriented programming language such as Smalltalk, C++or the like, and conventional procedural programming languages, such asthe “C” programming language or similar programming languages. Thecomputer readable program instructions may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider). In some embodiments, electronic circuitry including, forexample, programmable logic circuitry, field-programmable gate arrays(FPGA), or programmable logic arrays (PLA) may execute the computerreadable program instructions by utilizing state information of thecomputer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations (e.g., FIG. 2 and FIG. 3) and/or block diagrams(e.g., FIG. 1 and FIG. 5) of methods, apparatus (systems), and computerprogram products according to embodiments of the invention. It will beunderstood that each block of the flowchart illustrations and/or blockdiagrams, and combinations of blocks in the flowchart illustrationsand/or block diagrams, can be implemented by computer readable programinstructions (e.g., program code 514).

These computer readable program instructions may be provided to aprocessor (e.g., CPU 502) of a general purpose computer, special purposecomputer, or other programmable data processing apparatus (e.g.,computer 102) to produce a machine, such that the instructions, whichexecute via the processor of the computer or other programmable dataprocessing apparatus, create means for implementing the functions/actsspecified in the flowchart and/or block diagram block or blocks. Thesecomputer readable program instructions may also be stored in a computerreadable storage medium (e.g., computer data storage unit 512) that candirect a computer, a programmable data processing apparatus, and/orother devices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions (e.g., program code 514) mayalso be loaded onto a computer (e.g. computer 102), other programmabledata processing apparatus, or other device to cause a series ofoperational steps to be performed on the computer, other programmableapparatus or other device to produce a computer implemented process,such that the instructions which execute on the computer, otherprogrammable apparatus, or other device implement the functions/actsspecified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

While embodiments of the present invention have been described hereinfor purposes of illustration, many modifications and changes will becomeapparent to those skilled in the art. Accordingly, the appended claimsare intended to encompass all such modifications and changes as fallwithin the true spirit and scope of this invention.

What is claimed is:
 1. A method of selecting and displaying one or moreimages, the method comprising the steps of: a computer identifying auser and user profile information corresponding to the identified user;the computer identifying a sentiment of the user; based on the userprofile information, the computer determining an association between theidentified sentiment of the user and one or more sentiments included ina plurality of sentiments conveyed by a plurality of images; thecomputer determining that one or more images included in the pluralityof images convey the one or more sentiments; based on the associationbetween the identified sentiment of the user and the one or moresentiments and the one or more images conveying the one or moresentiments, the computer selecting the one or more images from theplurality of images; the computer displaying the selected one or moreimages, wherein the computer is included in a digital picture frame,wherein the step of displaying the selected one or more images includesdisplaying the one or more images on a display included in the digitalpicture frame, and wherein the step of identifying the sentiment of theuser includes the steps of: the computer determining that the user is ina proximity to the digital picture frame; in response to the step ofdetermining that the user in the proximity to the digital picture frame,the computer receiving lighting information from a light sensor coupledto the digital picture frame, wherein the lighting information includesa measurement of ambient lighting of a room in which the digital pictureframe is located; and in response to the step of determining that theuser is in the proximity to the digital picture frame, and based on themeasurement of the ambient lighting of the room in which the digitalpicture frame is located, the computer determining an emotional state ofthe user, the emotional state being included in the sentiment of theuser, wherein the step of selecting the one or more images is based inpart on the measurement of the ambient lighting of the room in which thedigital picture frame is located; the computer determining that one ormore additional users are in the proximity to the digital picture frame;in response to the one or more additional users being in the proximityto the digital picture frame, the computer determining one or moreemotional states of the one or more additional users, respectively;based on the emotional state of the user and the one or more emotionalstates of the one or more additional users, the computer determining oneemotional state of a majority of users in a group of users consisting ofthe user and the one or more additional users; the computer determiningan association between the one emotional state of the majority of theusers and one sentiment included in the plurality of sentiments conveyedby the plurality of images; the computer determining that an imageincluded in the plurality of images conveys the one sentiment; based onthe association between the one emotional state of the majority of usersand the one sentiment and based on the image conveying the onesentiment, the computer selecting the image from the plurality ofimages; and the computer displaying the image on the display included inthe digital picture frame.
 2. The method of claim 1, further comprisingthe step of the computer collecting schedules of events of interest tothe user from a calendar service or a calendar application, wherein thestep of identifying the sentiment of the user is based on the schedulesof events of interest to the user collected from the calendar service orthe calendar application.
 3. The method of claim 1, further comprisingthe step of the computer automatically generating metadata tags for theplurality of images, the metadata tags specifying respective contents ofcorresponding images included in the plurality of images, wherein thestep of determining that the one or more images convey the one or moresentiments includes: identifying contents of the one or more imagesbased on one or more metadata tags generated for the one or more images;and identifying the one or more sentiments based on the identifiedcontents of the one or more images.
 4. The method of claim 1, furthercomprising the step of the computer automatically generating metadatatags for the plurality of images, the metadata tags specifyingrespective sentiments of corresponding images included in the pluralityof images, wherein the step of determining that the one or more imagesconvey the one or more sentiments includes identifying the one or moresentiments based on one or more metadata tags generated for the one ormore images.
 5. The method of claim 1, further comprising the step ofthe computer identifying a new post authored by the user via a socialmedia service, wherein the step of identifying the sentiment of the userincludes determining the sentiment based in part on contents of the newpost, and wherein the step of selecting the one or more images is basedin part on the sentiment based in part on the contents of the new post.6. The method of claim 1, further comprising the steps of: based on thelighting information received from the light sensor coupled to thedigital picture frame, the computer determining that the ambientlighting in the room is dim; based on the ambient lighting in the roombeing dim, the computer determining a mood of an environment inproximity to the digital picture frame is somber; based on the mood ofthe environment in proximity to the digital picture frame being somber,the computer determining an association between the mood of theenvironment in proximity to the digital picture frame and a sentimentincluded in the plurality of sentiments conveyed by the plurality ofimages; the computer determining that an image included in the pluralityof images conveys the sentiment; based on the association between themood of the environment in proximity to the digital picture frame andthe sentiment and based on the image conveying the sentiment, thecomputer selecting the image from the plurality of images; and thecomputer displaying the selected image on the display included in thedigital picture frame.
 7. The method of claim 1, further comprising thestep of: providing at least one support service for at least one ofcreating, integrating, hosting, maintaining, and deployingcomputer-readable program code in the computer, the program code beingexecuted by a processor of the computer to implement the steps ofidentifying the user and the user profile information, identifying thesentiment of the user, determining the association between theidentified sentiment of the user and the one or more sentiments,determining that the one or more images convey the one or moresentiments, selecting the one or more images, displaying the selectedone or more images, displaying the one or more images, determining thatthe user is in the proximity to the digital picture frame, receiving thelighting information, determining the emotional state of the user,determining that the one or more additional users are in the proximityto the digital picture frame, determining the one or more emotionalstates of the one or more additional users, determining the oneemotional state of the majority of users in the group of users,determining the association between the one emotional state and the onesentiment, determining that the image conveys the one sentiment,selecting the image from the plurality of images, and displaying theimage on the display.
 8. A computer program product, comprising: acomputer-readable, storage device; and a computer-readable program codestored in the computer-readable, storage device, the computer-readableprogram code containing instructions that are executed by a centralprocessing unit (CPU) of a computer system to implement a method ofselecting and displaying one or more images, the method comprising thesteps of: the computer system identifying a user and user profileinformation corresponding to the identified user; the computer systemidentifying a sentiment of the user; based on the user profileinformation, the computer system determining an association between theidentified sentiment of the user and one or more sentiments included ina plurality of sentiments conveyed by a plurality of images; thecomputer system determining that one or more images included in theplurality of images convey the one or more sentiments; based on theassociation between the identified sentiment of the user and the one ormore sentiments and the one or more images conveying the one or moresentiments, the computer system selecting the one or more images fromthe plurality of images; the computer system displaying the selected oneor more images, wherein the computer system is included in a digitalpicture frame, wherein the step of displaying the selected one or moreimages includes displaying the one or more images on a display includedin the digital picture frame, and wherein the step of identifying thesentiment of the user includes the steps of: the computer systemdetermining that the user is in a proximity to the digital pictureframe; in response to the step of determining that the user in theproximity to the digital picture frame, the computer system receivinglighting information from a light sensor coupled to the digital pictureframe, wherein the lighting information includes a measurement ofambient lighting of a room in which the digital picture frame islocated; and in response to the step of determining that the user is inthe proximity to the digital picture frame, and based on the measurementof the ambient lighting of the room in which the digital picture frameis located, the computer system determining an emotional state of theuser, the emotional state being included in the sentiment of the user,wherein the step of selecting the one or more images is based in part onthe measurement of the ambient lighting of the room in which the digitalpicture frame is located; the computer system determining that one ormore additional users are in the proximity to the digital picture frame;in response to the one or more additional users being in the proximityto the digital picture frame, the computer system determining one ormore emotional states of the one or more additional users, respectively;based on the emotional state of the user and the one or more emotionalstates of the one or more additional users, the computer systemdetermining one emotional state of a majority of users in a group ofusers consisting of the user and the one or more additional users; thecomputer system determining an association between the one emotionalstate of the majority of the users and one sentiment included in theplurality of sentiments conveyed by the plurality of images; thecomputer system determining that an image included in the plurality ofimages conveys the one sentiment; based on the association between theone emotional state of the majority of users and the one sentiment andbased on the image conveying the one sentiment, the computer systemselecting the image from the plurality of images; and the computersystem displaying the image on the display included in the digitalpicture frame.
 9. The computer program product of claim 8, wherein themethod further comprises the step of the computer system collectingschedules of events of interest to the user from a calendar service or acalendar application, wherein the step of identifying the sentiment ofthe user is based on the schedules of interest to the user collectedfrom the calendar service or the calendar application.
 10. The computerprogram product of claim 8, wherein the method further comprises thestep of the computer system automatically generating metadata tags forthe plurality of images, the metadata tags specifying respectivecontents of corresponding images included in the plurality of images,wherein the step of determining that the one or more images convey theone or more sentiments includes: identifying contents of the one or moreimages based on one or more metadata tags generated for the one or moreimages; and identifying the one or more sentiments based on theidentified contents of the one or more images.
 11. The computer programproduct of claim 8, wherein the method further comprises the step of thecomputer system automatically generating metadata tags for the pluralityof images, the metadata tags specifying respective sentiments ofcorresponding images included in the plurality of images, wherein thestep of determining that the one or more images convey the one or moresentiments includes identifying the one or more sentiments based on oneor more metadata tags generated for the one or more images.
 12. Thecomputer program product of claim 8, wherein the method furthercomprises the steps of: based on the lighting information received fromthe light sensor coupled to the digital picture frame, the computersystem determining that the ambient lighting in the room is dim; basedon the ambient lighting in the room being dim, the computer systemdetermining a mood of an environment in proximity to the digital pictureframe is somber; based on the mood of the environment in proximity tothe digital picture frame being somber, the computer system determiningan association between the mood of the environment in proximity to thedigital picture frame and a sentiment included in the plurality ofsentiments conveyed by the plurality of images; the computer systemdetermining that an image included in the plurality of images conveysthe sentiment; based on the association between the mood of theenvironment in proximity to the digital picture frame and the sentimentand based on the image conveying the sentiment, the computer systemselecting the image from the plurality of images; and the computersystem displaying the selected image on the display included in thedigital picture frame.
 13. A computer system comprising: a centralprocessing unit (CPU); a memory coupled to the CPU; and a computerreadable storage device coupled to the CPU, the computer readablestorage device containing instructions that are executed by the CPU viathe memory to implement a method of selecting and displaying one or moreimages, the method comprising the steps of: the computer systemidentifying a user and user profile information corresponding to theidentified user; the computer system identifying a sentiment of theuser; based on the user profile information, the computer systemdetermining an association between the identified sentiment of the userand one or more sentiments included in a plurality of sentimentsconveyed by a plurality of images; the computer system determining thatone or more images included in the plurality of images convey the one ormore sentiments; based on the association between the identifiedsentiment of the user and the one or more sentiments and the one or moreimages conveying the one or more sentiments, the computer systemselecting the one or more images from the plurality of images; thecomputer system displaying the selected one or more images, wherein thecomputer system is included in a digital picture frame, wherein the stepof displaying the selected one or more images includes displaying theone or more images on a display included in the digital picture frame,and wherein the step of identifying the sentiment of the user includesthe steps of: the computer system determining that the user is in aproximity to the digital picture frame; in response to the step ofdetermining that the user in the proximity to the digital picture frame,the computer system receiving lighting information from a light sensorcoupled to the digital picture frame, wherein the lighting informationincludes a measurement of ambient lighting of a room in which thedigital picture frame is located; and in response to the step ofdetermining that the user is in the proximity to the digital pictureframe, and based on the measurement of the ambient lighting of the roomin which the digital picture frame is located, the computer systemdetermining an emotional state of the user, the emotional state beingincluded in the sentiment of the user, wherein the step of selecting theone or more images is based in part on the measurement of the ambientlighting of the room in which the digital picture frame is located; thecomputer system determining that one or more additional users are in theproximity to the digital picture frame; in response to the one or moreadditional users being in the proximity to the digital picture frame,the computer system determining one or more emotional states of the oneor more additional users, respectively; based on the emotional state ofthe user and the one or more emotional states of the one or moreadditional users, the computer determining one emotional state of amajority of users in a group consisting of the user and the one or moreadditional users; the computer system determining an association betweenthe one emotional state of the majority of the users and one sentimentincluded in the plurality of sentiments conveyed by the plurality ofimages; the computer system determining that an image included in theplurality of images conveys the one sentiment; based on the associationbetween the one emotional state of the majority of users and the onesentiment and based on the image conveying the one sentiment, thecomputer system selecting the image from the plurality of images; andthe computer system displaying the image on the display included in thedigital picture frame.
 14. The computer system of claim 13, wherein themethod further comprises the step of the computer system collectingschedules of events of interest to the user from a calendar service or acalendar application, wherein the step of identifying the sentiment ofthe user is based on the schedules of events of interest to the usercollected from the calendar service or the calendar application.
 15. Thecomputer system of claim 13, wherein the method further comprises thestep of the computer system automatically generating metadata tags forthe plurality of images, the metadata tags specifying respectivecontents of corresponding images included in the plurality of images,wherein the step of determining that the one or more images convey theone or more sentiments includes: identifying contents of the one or moreimages based on one or more metadata tags generated for the one or moreimages; and identifying the one or more sentiments based on theidentified contents of the one or more images.
 16. The computer systemof claim 13, wherein the method further comprises the step of thecomputer system automatically generating metadata tags for the pluralityof images, the metadata tags specifying respective sentiments ofcorresponding images included in the plurality of images, wherein thestep of determining that the one or more images convey the one or moresentiments includes identifying the one or more sentiments based on oneor more metadata tags generated for the one or more images.
 17. Thecomputer system of claim 13, wherein the method further comprises thesteps of: based on the lighting information received from the lightsensor coupled to the digital picture frame, the computer systemdetermining that the ambient lighting in the room is dim; based on theambient lighting in the room being dim, the computer system determininga mood of an environment in proximity to the digital picture frame issomber; based on the mood of the environment in proximity to the digitalpicture frame being somber, the computer system determining anassociation between the mood of the environment in proximity to thedigital picture frame and a sentiment included in the plurality ofsentiments conveyed by the plurality of images; the computer systemdetermining that an image included in the plurality of images conveysthe sentiment; based on the association between the mood of theenvironment in proximity to the digital picture frame and the sentimentand based on the image conveying the sentiment, the computer systemselecting the image from the plurality of images; and the computersystem displaying the selected image on the display included in thedigital picture frame.